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[57] ABSTRACT 

The present invention is embodied in a charge coupled 
device (CCD)/charge injection device (CID) architecture 
capable of performing a Fourier transform by simultaneous 
matrix vector multiplication (MVM) operations in respec- 
tive plural CCD/CID arrays in parallel in 0(1) steps. For 
example, in one embodiment, a first CCD/CID array stores 
charge packets representing a first matrix operator based 
upon permutations of a Hartley transform and computes the 
Fourier transform of an incoming vector. A second CCD/ 
CID array stores charge packets representing a second 
matrix operator based upon different permutations of a 
Hartley transform and computes the Fourier transform of an 
incoming vector. The incoming vector is applied to the 
inputs of the two CCD/CID arrays simultaneously, and the 
real and imaginary parts of the Fourier transform are pro- 
duced simultaneously in the time required to perform a 
single MVM operation in a CCD/CID array. 

12 Claims, 8 Drawing Sheets 



i 




130 


U.S. Patent 


Sep. 14, 1999 


Sheet 1 of 8 


5,952,685 












FIG. 2 C (PRIOR art) FIG. 2D (PRIOR art) FIG. 2E (prior art) 












Sheet 5 of 8 


5,952,685 


U.S. Patent Sep. 14,1999 


CC 

1 

3 


o 

s* 


a: 

LU 

Q 

Q 

< 


a: 

Lt_l 

Q 

O 

< 


LlJ 

t/j 

o 

LiJ 

le 

> 


i_ a: 

by 

ie| 

feo 

o 



o 

3^ 

<ti 

5g 



FIG. 







U.S. Patent 


Sep. 14, 1999 


Sheet 7 of 8 


5,952,685 



FIG. 10 




U.S. Patent 


Sep. 14, 1999 Sheet 8 of 8 


5,952,685 





5 , 952,685 


1 

SIGNAL PROCESSING APPLICATIONS OF 
MASSIVELY PARALLEL CHARGE DOMAIN 
COMPUTING DEVICES 

CROSS-REFERENCE TO RELATED 
APPLICATIONS 

This is a division, of application Ser. No. 08/161,908, filed 
Nov. 30, 1993, now U.S. Pat. No. 5,508,538, which is a 
continuation-in-part of application Ser. No. 08/049,829, 
filed Apr. 19, 1993, now U.S. Pat. No. 5,491,650. 

ORIGIN OF THE INVENTION 

The invention described herein was made in the perfor- 
mance of work under a NASA contract, and is subject to the 
provisions of Public Law 96-517 (35 USC 202) in which the 
contractor has elected to retain title. 

BACKGROUND OF THE INVENTION 

1. Technical Field 

The invention relates to charge coupled device (CCD) and 
charge injection device (CID) hardware applied to higher 
precision parallel arithmetic processing devices particularly 
adapted to perform large numbers of multiply- accumulate 
operations with massive parallelism and to methods for 
performing signal processing including Fourier transforms 
and convolution. 

2. Background Art 

Many algorithms required for scientific modeling make 
frequent use of a few well defined, often functionally simple, 
but computationally very intensive data processing opera- 
tions. Those operations generally impose a heavy burden on 
the computational power of a conventional general-purpose 
computer, and run much more efficiently on special-purpose 
processors that are specifically tuned to address a single 
intensive computation task only. A typical example among 
the important classes of demanding computations are vector 
and matrix operations such as multiplication of vectors and 
matrices, solving linear equations, matrix inversion, eigen- 
value and eigenvector search, etc. Most of the computation- 
ally more complex vector and matrix operations can be 
reformulated in terms of basic matrix-vector and matrix — 
matrix multiplications. From a neural network perspective, 
the product of the synaptic matrix by the vector of neuron 
potentials is another good example. 

An innovative hybrid, analog-digital charge -domain 
technology, for the massively parallel VLSI implementation 
of certain large scale matrix-vector operations, has recently 
been developed, as disclosed in U.S. Pat. No. 5,054,040. It 
employs arrays of Charge Coupled/Charge Injectioned 
Device (CCD/CID) cells holding an analog matrix of charge, 
which process digital vectors in parallel by means of binary, 
non-destructive charge transfer operations. FIG. 1 shows a 
simplified schematic of the CCD/CID array 100. Each cell 
110 in the array 100 connects to an input column line 120 
and an output row line 130 by means of a column gate 140 
and a row gate 150. The gates 140, 150 hold a charge packet 
160 in the silicon substrate underneath them that represents 
an analog matrix element. The matrix charge packets 160 are 
initially stored under the column gates 140. In the basic 
matrix-vector multiplication (hereinafter referred to as 
“MVM”) mode of operation, for binary input vectors, the 
matrix charge packets 160 are transferred from under the 
column gate 140 toward the row gates 150 only if the input 
bit of the column indicates a binary ‘one’. The charge 
transferred under the row gates 150 is summed capacitively 


2 

on each output row line 130, yielding an analog output 
vector which is the product of the binary input vector with 
the analog charge matrix. By virtue of the CCD/CID device 
physics, the charge sensing at the row output lines 130 is of 
5 a non-destructive nature, and each matrix charge packet 160 
is restored to its original state simply by pushing the charge 
back under its column gate 140. 

FIG. 2 is an illustration of the binary-analog MVM 
computation cycle for a single row of the CCD/CID array. 
io In FIG. 2 A, the matrix charge packet 160 sits under the 
column gate 140. In FIG. 2B, the row line 130 is reset to a 
reference voltage. In FIG. 2C, if the column line 120 
receives a logic one input bit, the charge packet 160 is 
transferred underneath the row gate 150. In FIG. 2D, the 
15 transferred charge packet 160 is sensed capacitively by a 
charge in voltage on the output row line 130. In FIG. 2E the 
charge packet 160 is returned under the column gate 140 in 
preparation for the next cycle. A bit-serial digital-analog 
MVM can be obtained from a sequence of binary-analog 
20 MVM operations, by feeding in successive vector input bits 
sequentially, and adding the corresponding output contribu- 
tions after scaling them with the appropriate powers of two. 
A simple parallel array of divide-by-two circuits at the 
output accomplishes this task. Further extensions of the 
25 basic MVM scheme of FIG. 1 support full digital outputs by 
parallel A/D conversion at the outputs, and four-quadrant 
operation by differential circuit techniques. 

FIG. 3 illustrates how the matrix charge packets are 
loaded into the array. In FIG. 3A, appropriate voltages are 
30 applied to each gate 170 in each cell 110 of the CCD/CID 
array 100 so as to configure each cell 110 as a standard 
4-phase CCD analog shift register to load all of the cells 110 
sequentially. In FIG. 3B, the same gates 170 are used for row 
and column charge transfer operations as described above 
35 with reference to FIG. 2. 

Signal Processing 

The foundation of conventional signal processing algo- 
rithms is based on the use of fast techniques for performing 
40 various discrete transformations such as the discrete Fourier 
transform (DFT), discrete sin transform (DST), discrete 
co-sine transform (DCT), discrete Hartley transform (DHT), 
and others. Consider the discrete Fourier Transform (DFT). 
The DFT can be represented by a Matrix-vector Multipli- 
45 cation (MVM) with a computational complexity of 0(N 2 ). 
However, for both serial and parallel computation on con- 
ventional hardware, the Fast Fourier Transform (FFT) is 
always preferred. 

For serial computations, the FFT achieves a computa- 
50 tional complexity of O(NLogN). Also, for implementation 
on parallel and vector computer architectures, the FFT has 
been considered as the base line algorithm. In particular, 
with O(N) processors, a time lower bond of O(LogN) can be 
achieved in computing the FFT. Note, however, that this 
55 result is more of a theoretical importance than a practical one 
since, particularly for large N, implementation of the algo- 
rithm to achieve the above time lower bound would require 
an architecture with an excessive number of processors, and, 
more importantly, a very complex processors interconnec- 
60 tion structure. 

With conventional hardware technology, the time lower 
bound in computing a MVM is O(LogN) by using 0(N 2 ) 
processors. This result is more relevant to theory than to 
practice, since such an implementation of MVM requires a 
65 very complex parallel architecture. 

In contrast, a practical implementation of MVM on the 
CCD/CID chip can be performed in 0(1) steps. This indi- 
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cates that, for efficient implementation of signal processing 
applications on CCD/CID chips, a new algorithmic frame- 
work is required, which significantly differs from the con- 
ventional fast techniques framework. In particular, the DHT 
can be more efficiently implemented than the FHT. In fact, 
while the DHT can be performed in 0(1) with one CCD/CID 
chip, the implementation of FHT requires O(LogN) chips 
and takes O(LogN) steps. 

Accordingly, there is a need for a massively parallel 
charge domain computing device and process which 
employs the DHT to achieve massive parallelism in signal 
processing in order to fully exploit the advantages of the 
CCD/CID architecture. In particular, there is a need for a 
process to perform a Fourier transform in a single MVM 
operation or plural simultaneous MVM operations in paral- 
lel in a CCD/CID architecture, each operation being per- 
formed in 0(1) steps. There is also a need for a process to 
perform a convolution in a single MVM operation in a 
CCD/CID architecture in 0(1) steps, convolution. 

SUMMARY OF THE DISCLOSURE 

The present invention is embodied in a CCD/CID archi- 
tecture capable of performing a Fourier transform by simul- 
taneous MVM operations in respective plural CCD/CID 
arrays in parallel in 0(1) steps. 

In one embodiment, a first CCD/CID array stores charge 
packets representing a first matrix operator based upon 
permutations mutations of a Hartley transform and computes 
the real part of the Fourier transform of an incoming vector. 
A second CCD/CID array stores charge packets representing 
a second matrix operator based upon different permutations 
of a Hartley transform and computes the real part of the 
Fourier transform of an incoming vector. The incoming 
vector is applied to the inputs of the two CCD/CID arrays 
simultaneously, and the real and imaginary parts of the 
Fourier transform are produced simultaneously in the time 
required to perform a single MVM operation in a CCD/CID 
array. 

In another embodiment, parallel MVM operations com- 
pute a Fourier transform using CCD/CID arrays of a fraction 
of the size of the transform itself. In this latter embodiment, 
the input signal vector is bifurcated into smaller vectors 
whose sums and differences are transformed by different 
permutations of a fractional-sized Hartley transform to pro- 
duce respective real and imaginary parts of the desired 
Fourier transform. Each permutation of a Hartley transform 
is embedded in a different CCD/CID array or chip and all 
CCD/CID arrays receive their respective fractional portions 
of the input signal vector simultaneously and perform 
respective MVM operations simultaneously. 

In one implementation of this latter embodiment, the 
CCD/CID array size is half the size of the incoming signal 
vector, the incoming signal vector is bifurcated into two 
equal portions. Two of four different permutations of a 
Hartley transform embedded in four respective CCD/CID 
arrays receive the sum of the bifurcated incoming signal 
portions to produce, respectively, the real and imaginary 
parts of the even terms of the desired Fourier transform. The 
remaining two permutations of the Hartley transform receive 
the difference between the two incoming signal portions to 
produce, respectively, the real and imaginary parts of the odd 
terms of the desired Fourier transform. 

The present invention also embodied in a CCD/CID 
architecture capable of performing a convolution of two 
signal vectors in a single MVM operation in a CCD/CID 
array in 0(1) steps. In one implementation, one of the two 
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signal vectors is known beforehand and even and odd 
permutations of its Hartley transformed are precomputed. A 
convolution matrix operator is constructed by the matrix 
multiplication of a Hartley transform matrix by the sum of 
5 the even and odd permutations of the Hartley transform of 
the known signal vector, the result being matrix-multiplied 
in turn by a Hartley transform matrix. The resulting matrix 
is embedded in a single CCD/CID array of charge packets. 
The unknown signal vector is input to the CCD/CID array to 
10 produce a vector equal to the convolution of the two signal 
vectors in a single MVM operation. 

The present invention is preferably implemented in a 
CCD/CID MVM processor of the type described in the 
above-referenced parent application which stores each bit of 
!5 each matrix element as a separate CCD charge packet. The 
bits of each input vector are separately multiplied by each bit 
of each matrix element in massive parallelism and the 
resulting products are combined appropriately to synthesize 
the correct product. In one embodiment, the CCD/CID 
20 MVM array is a single planar chip in which each matrix 
element occupies a single column of b bits, b being the bit 
resolution, there being N rows and N columns of such single 
columns in the array. In another embodiment, the array 
constitutes a stack of b chips, each chip being a bit -plane and 
25 storing a particular significant bit of all elements of the 
matrix. In this second embodiment, an output chip is con- 
nected edge-wise to the bit-plane chips and performs the 
appropriate arithmetic combination steps. 

In a preferred embodiment, the MVM processor of the 
30 invention includes an array of N rows and M columns of 
CCD matrix cell groups corresponding to a matrix of N rows 
and M columns of matrix elements, each of the matrix 
elements representable with b binary bits of precision, each 
of the matrix cell groups including a column of b CCD cells 
35 storing b CCD charge packets representing the b binary bits 
of the corresponding matrix element, the amount of charge 
in each packet corresponding to one of two predetermined 
amounts of charge. Each of the CCD cells includes a holding 
site and a charge sensing site, each charge packet initially 
40 residing at the respective holding site. The MVM processor 
further includes a device for sensing, for each row, an analog 
signal corresponding to a total amount of charge residing 
under all charge sensing sites of the CCD cells in the row, 
an array of C rows and M columns CCD vector cells 
45 corresponding to a vector of M elements representable with 
c binary bits of precision, each one of the M columns of 
CCD vector cells storing a plurality of c charge packets 
representing the c binary bits of the corresponding vector 
element, the amount of charge in each packet corresponding 
50 to one of two predetermined amount of charge. A multiply- 
ing device operative for each one of the c rows of the CCD 
vector cells temporarily transfers to the charge sensing site 
the charge packet in each one of the M columns of matrix 
cells for which the charge packet in the corresponding one 
55 of the M columns and the one row of the CCD vector cells 
has an amount of charge corresponding to a predetermined 
binary value. 

The preferred embodiment further includes an arithmetic 
processor operative in synchronism with the multiplying a 
60 device including a device for receiving, for each row, the 
sensed signal, whereby to receive Nxb signals in each one 
of c operations of the multiplying a device, a device for 
converting each of the signals to a corresponding byte of 
output binary bits of all the signals in accordance with 
65 appropriate powers of two to generate bits representing an 
N-element vector corresponding to the product of the vector 
and the matrix. 
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In one embodiment, the array of matrix CCD cells is 
distributed among a plurality of b integrated circuits con- 
taining sub-arrays of the M columns and N rows of the 
matrix CCD cells, each of the sub-arrays corresponding to a 
bit -plane of matrix cells representing bits of the same power 5 
of two for all of the matrix elements. A backplane integrated 
circuit connected edgewise to all of the b integrated circuits 
includes a device for associating respective rows of the 
vector CCD elements with respective rows of the matrix 
CCD elements, whereby the multiplying device operates on 10 
all the rows of the vector CCD elements in parallel. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a simplified diagram of a CCD/CID MVM 
processor array of the prior art. 

FIGS. 2A, 2B, 2C, 2D and 2E illustrate a sequence of 
matrix-vector multiply operations in a unit cell of the array 
of FIG. 1. 

FIGS. 3Aand 3B illustrate, respectively, electronic load- 20 
ing of the matrix elements and arithmetic operations in a unit 
cell of the array of FIG. 1. 

FIG. 4 is a plan view of a preferred CCD/CID MVM 
processor employed in carrying out the present invention. 

FIG. 5 is a schematic diagram of a typical architecture for 25 
a higher precision arithmetic processor employed in com- 
bination with the CCD/CID processor of FIG. 4. 

FIG. 6 is a diagram of a three-dimensional embodiment of 
the CCD/CID MVM processor of FIG. 4. 

FIG. 7 is a block diagram illustrating a discrete Hartley 
transform process employed in the invention. 

FIG. 8 is a block diagram illustrating an inverse discrete 
Hartley transform employed in carrying out the invention. 

FIG. 9 is a diagram of a permutation matrix employed in 35 
carrying out the invention. 

FIG. 10 is a block diagram of a CCD/CID architecture for 
carrying out the discrete Fourier transform process of the 
present invention. 

4D 

FIG. 11 is a block diagram of a CCD/CID architecture for 
carrying out the convolution process of the present inven- 
tion. 

FIG. 12 is a block diagram of a decimation-in-frequency 
fast Hartley transform process employed in a process of the 45 
invention. 

FIG. 13 is a block diagram of a CCD/CID architecture for 
carrying out an area-efficient discrete Fourier transform 
process of the present invention. 

50 

DETAILED DESCRIPTION OF THE 
PREFERRED EMBODIMENTS 

Preferred CCD MVM Processor Structure 

In order to achieve high precision in a CCD/CID MVM 55 
processor, the present invention can employ an advanced 
architecture described in the above-referenced parent appli- 
cation which is illustrated schematically in FIGS. 4 and 5. In 
the following, the architecture is described in its basic form. 
However, different architectures can be derived from this 60 
basic form which are not discussed here. A key element of 
the CCD/CID MVM array 200 of FIGS. 4 and 5 is the 
encoding in each CCD/CID processor cell 210 of one bit of 
the binary representation of each matrix element. As shown 
in FIG. 4, if a matrix A is to be specified with b bits of 65 
precision, each element A iy of the matrix occupies a single 
column 205 of b cells, namely cells corresponding to [b(i- 
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l)+l,j], for 1=1, . . . b, labelled in FIG. 4 as A 0 ,-,-, . . . A b ~ 1 iJ : 
Thus, the CCD/CID MVM processor array 200 of FIG. 4 is 
an array of N columns and N rows of matrix elements, each 
matrix element itself being a column 205 of b CCD/CID 
cells 210. There are, therefore, at total of NxNxb cells 210 
in the array 200. A vector of N elements representable with 
a precision of b binary bits is stored in an array 230 of 
CCD/CID cells 235, the array 230 being organized in N 
columns 240 of b CCD/CID cells, each cell 235 in a given 
column 240 storing the corresponding one of the b binary 
bits of the corresponding vector element. Each CCD/CID 
cell 210 in FIG. 4 is of the type described above with 
reference to FIGS. 1-3 and is operated in the same manner 
with N column input lines (coupled to a successive row of 
N vector CCD/CID cells 235 in the vector CCD/CID array 
230) and row output lines of the type illustrated in FIG. 1. 

Matrix- vector multiplication will now be described with 
reference to the example of a conventional matrix-vector 
product obtained by adding the products of the elements in 
the vector and each row of the matrix. Of course, the present 
invention is not confined to a particular type of matrix vector 
product, and can be used to compute other types of matrix- 
vector products. One example of another well-known type 
of matrix-vector product is that obtained by adding the 
products of a respective vector element multiplied by all 
elements in a respective column of the matrix. 

Computation proceeds as follows. At clock cycle one, the 
matrix A, in its binary representation, is multiplied by the 
binary vector labelled u° 1 , . . . u 0 ^, which contains the least 
significant bits of u 1? . . . u^ (i.e., by the top row of charge 
packets in the array 230 of vector CCD/CID cells 235). By 
virtue of the charge transfer mechanism, analog voltages 
labelled are sensed at the output of each 

one of the bxN rows of the matrix array 200. To keep track 
of the origin of this contribution to the result, a left super- 
script (0) v is utilized in the notation employed herein. 

In the present example, all of the products computed in 
the array 200 are synthesized together in accordance with 
corresponding powers of two to form the N elements of the 
transformed (output) vector. This is accomplished in an 
arithmetic processor illustrated in FIG. 5. The arithmetic 
processor is dedicated to the computation of the particular 
type of matrix-vector product computed in the present 
example. Processors other than that illustrated in FIG. 5 
could be employed in making these same computations from 
the products produced by the array 200, and the present 
invention is not confined to the type of arithmetic processor 
illustrated in FIG. 5. Of course, in order to compute matrix- 
vector products different from the type computed in the 
present example, an arithmetic processor different from that 
illustrated in FIG. 5 would be employed in combination with 
the array 200 of FIG. 4. 

At clock cycle two, the voltages sensed at each of the N 
rows are fed into respective pipelined A/D converters 300, 
each individual converter 300 having b^ bits of precision 
(where d=log 2 N, and N denotes the number of columns of 
A), while simultaneously A is multiplied by u 1 1 , . .. u 1 ^, (i.e., 
by the second row of charge packets in the array 230 of 
vector CCD/CID cells), yielding (1) v. 

At clock cycle three, the digital representations of ^v° l9 
are bit mapped into a V register 305 with an 
appropriate bit — offset toward the most significant bit posi- 
tion. Specifically, the result or element obtained during 
clock cycle k is offset in the appropriate V register by lk bits 
toward the most significant bit position (toward the leftmost 
bit) at clock cycle k. This offset is controlled by an offset 
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counter 310 (which is incremented by one at each new clock 
cycle). Next, the voltages (1) v are fed into the A/D converters 
300, and the vector u 2 1 , . . . u 2 ^ multiplies A to yield (2) v. 

Elements ( k \ l . with same row index i are then fed into 
cascaded sum circuits 315 shown in FIG. 5, in parallel for all 5 
i, and pipelined over k. The cascaded sum circuits 315 are 
connected in a binary tree architecture of the type well- 
known in the art. Hence, the components v t of the product 
v=Au are obtained after log 2 b cycles, and the overall latency 
is b+log 2 b+3. If one needs to multiply a set of vectors u by 10 
the same matrix A, this pipelined architecture will output a 
new result every b clock cycles. Clearly, a far higher 
precision has been achieved than was previously available. 

An added benefit is that the refresh time overhead is sig- 
nificantly reduced, since in the matrix representation of FIG. 15 
4 each electron charge packet only refers to a binary 
quantity. 

While FIG. 4 indicates that the NxbxN cells 210 of the 
array 200 are formed on a single planar silicon chip, the 
array 200 can instead be implemented using many chips 20 
connected together, using Z-plane technology, as illustrated 
in FIG. 6. In such an embodiment, it is preferable to have 
each chip 400 assigned to a particular bit plane, in which 
each chip is an N-by-N CCD/CID array of CCD/CID cells 
210 of the type illustrated in FIG. 1, which stores NxN bits 25 
or charge packets, which in the present invention, however, 
represent binary values only. The first chip 400-0 stores the 
least significant bits of all matrix elements of the NxN 
matrix, the second chip 400-1 storing the next least signifi- 
cant bits of all matrix elements, and so forth, and the last 30 
chip 400-(b-l) storing the most significant bits of all matrix 
elements. A backplane chip 405 implements the array 230 of 
N columns and b rows of vector CCD cells 235 of FIG. 4. 
The backplane chip is connected edge-wise to the column 
input lines of all of the bit-plane chips 400. This permits 35 
every one of the b rows of vector CCD/CID cells 235 to be 
input to column input lines of respective ones of the b 
bit -plane chips 400, greatly enhancing performance and 
reducing the latency of a given matrix-vector multiplication 
operation. The arithmetic processor of FIG. 5 could also be 40 
implemented on the backplane chip 405. The architecture of 
the arithmetic processor of FIG. 5 could also be imple- 
mented on the backplane chip 405. The architecture of the 
arithmetic processor would depend upon the type of matrix- 
vector product to be computed. The Z-plane embodiment of 45 
FIG. 6 permits all b bits of every matrix element to be 
multiplied by a given vector element, and therefore is 
potentially much faster than the embodiment of FIGS. 4 and 
5. 

SIGNAL PROCESSING CCD/CID DEVICES 50 
Overview 

The remainder of the specification first describes how a 
complex DFT can be obtained from the DHT by using 
CCD/CID chips. Then, a more interesting result is presented, 55 
regarding the convolution of two signals. It is shown that, by 
using CCD/CID chips, the convolution can be presented as 
a single MVM, and hence can be performed with the highest 
area and time efficiency. A process is presented for area and 
time efficient DFT where the size of transform is much larger 60 
than the size of the CCD/CID chip. Finally, the specification 
outlines possible architectures that should enable high pre- 
cision computing with charge-domain devices. 

Computing DFT from DHT 65 

In its preferred embodiment, the signal processing CCD/ 
CID architecture of the present invention employs the DHT 
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as the main kernel for implementation on the CCD/CID 
chip. The DHT and its inverse are given as 


^nm 

and 


1 7mm 

: —casl 

N N 


nm 

: NH nm = casln — 
N 


( 1 ) 


(2) 


The kernel for DHT and inverse DHT are shown in FIGS. 7 
and 8, respectively. In the above expressions 


cas(0)=cos(0)+sin(0) (3) 

The first issue is the computation of the DFT from the 
DHT. Let the DFT and DHT of a signal g be given as 


f=F[g] and h=H[g] 


where F and H are the DFT and DHT operators. In terms of 
the real and imaginary part of f it follows that 


f=Re\f\+i Im\f\=E[h]-iO[h] 

where E and O are even and odd operators defined as 


k+ * k n h n + h -n K + h N . n 

hn ~ h-n hn ~ hfj-n 


h + -h 

0[h\ = — > 0[h n ] = 


( 4 ) 

( 5 ) 

(6) 


Let h + denote the vector h with normal ordering of the 
elements, i.e., 


h + =[h 0 , K,... h N J (7) 

Also, let h“ denote the permutation vector h as 


h =[h 0 , Vi, ■ ■ ■ hj (8) 

We note that h + and h“ can be related through a permutation 
matrix P as 


h~=Ph + (9) 

where P is illustrated in FIG. 9. From Eqs. (5), (6) and (9), 
the expressions for E[h] and 0[h] are obtained as 

P+I P-I (10) 

E[h] = -j-h and - 0[h] = -^-h 

From Eqs. (4) and (10), f is obtained as 

p+i p-i (11) 

/ = h+ i h 

J 2 2 

or 
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-continued 
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■+ / p- 
— Hg+i—Hg 


The matrices 


7=HYI and Wg 

(14) 

2 f=F[ 2 g\ and 2 h=H[ 2 g\ 

(15) 

f=F[g\ and h=H[g] 

(16) 


p+i 


where F and H denote the DFT and DHT operators, respec- 
tively. 

io In accordance with ParsevaFs Theorem, 


and 


g=F- 1 { 1 f0 2 f}=F- 1 {Fl 1 g]OFl 2 gYi 


(17) 


are constant and can be precomputed. Therefore, Eq. (12) 
leads to a direct realization of a DFT, as illustrated in FIG. 
10. In FIG. 10, the incoming signal g is multiplied simul- 
taneously in two CCD/CID arrays 500, 510 of the type 
illustrated in FIGS. 4 and 5, corresponding, respectively, to 
the matrices 


and 


P-l 


-H. 


where F -1 is the inverse DFT operator and O indicates 
15 component-by-component multiplication of two vectors. 
The above expression forms the basis of conventional con- 
volution algorithms. Assuming 2 f to be known a priori, a first 
FFT is used to compute 1 f in O(NLogN) steps, then the 
product of 1 f0 2 f is obtained in O(N) steps, and finally an 
20 inverse FFT is used to compute g in O(NLogN) steps. 

However, Eq. (17) is not suitable for implementation on 
CCD/CID chips. This is because it involves more than just 
a single MVM operation. Specifically, it requires a first 
MVM operation to compute the forward Fourier transforms, 
25 followed by a component-by-component multiplication of 
two vectors followed by a second MVM operation to 
compute the inverse Fourier transform. Hence, a new for- 
mulation is needed to take into account the fact that the only 
operation that can be efficiently performed on a CCD/CID is 
30 a MVM. In terms of DHT, h can be expressed as 


Referring to FIG. 4, the CCD/CID array 500 stores charge 
packets corresponding to binary values of 


h=V2[ 1 h + 0 2 h + - 1 h-Q 2 h-+ 1 h + Q 2 h-+ 1 h-Q 2 h + ] (18) 

One can also express h in terms of even and odd operators 
as 


h=E[ 2 h + ]0 1 h + +0[ 2 h + ]0 1 h~ 


in respective CCD/CID cells 235 arranged as illustrated in 40 or 
FIG. 4. 

The CCD/CID array 510 stores charge packets correspond- 
ing to binary values of 


h=E[ 2 h + ]0 1 h + +0[ 2 h + ]Q[P 1 h +] 


(19) 


P-l 


The O operation of two vectors can be described by a 
45 diagonal matrix-vector multiplication as 


in respective CCD/CID cells 235 arranged as illustrated in 
FIG. 4. Then, binary values of the vector g are input to each 
of the CCD/CID arrays 500, 510 in the manner described 
above with reference to FIGS. 4 and 5. The resulting output 
of the CCD/CID array 500 is a vector Re[f] which is the real 
part of the Fourier transform of g while the resulting output 
of the CCD/CID array 510 is a vector Im[f] which is the 
imaginary part of the Fourier transform of g. Thus, a 
complete Fourier transform is computed in the time required 
to perform a single MVM operation. 

Fast Area-Efficient Convolution Using DHT 

The convolution problem, that is, the computation of a 
signal g is defined as: 

g= 1 g* 2 g (13) 

where it is assumed that 1 g is the input and 2 g is known 
beforehand, and hence its DHT can be precomputed. Let 


v= 1 v0 2 v^v= 1 v 2 v (20) 

where 1 v is a diagonal matrix whose diagonal elements are 
those of the vector 1 v. Using such a representation, Eq. (18) 
can now be expressed as 

h={E[ 2 h + ]+d[ 2 h + ¥>yh + ( 21 ) 

A matrix Q is now defined in terms of the matrices E and 6: 

Q={E\ 2 h + t>?h + ^P} (22) 

Note that, since it is assumed that 2 g is known a priori, the 
matrix Q is also known a priori. From Eqs. (19) and (20), it 
follows that 

h=QH 1 g (23) 


50 


55 


60 


65 


and finally 
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g=H~ 1 h=NHQH 1 g (24) 

Eq. (24) is a convolution that can be performed in terms of 
a simple matrix-vector multiplication, by defining an appro- 
priate convolution operator. Specifically, we set 


C=NHQH (25) 

10 

as the convolution operator. As illustrated in FIG. 11, a 
CCD/CID array 600 of the type illustrated in FIGS. 4 and 5 
stores charge packets corresponding to the binary values of 
the matrix C and arranged in individual CCD/CID cells 235 
as illustrated in FIG. 4. Binary values of a vector 1 g are input 15 
to the CCD/CID array 600 in the manner described with 
reference to FIGS. 4 and 5. The CCD/CID array 600 
produces a vector g which is the convolution 1 g* 2 g of 
Equation 13. 

Computing a Large DFT with Small Size CCD/CID Chips 20 

We now consider the case wherein the size of the desired 
transform, N, is larger than the size, M, of the CCD/CID 
chip. A direct (“brute force”) approach would be to build an 
NxN DHT matrix by using MxM CCD/CID chips. From the 
foregoing discussion, it follows that 2(N/M ) 2 CCD/CID 25 
chips of size MxM would be required for computation of a 
DFT of size N. 

The main issues in devising a better approach for per- 
forming large DFT with small size CCD/CID chips can be 
summarized as follows: 30 

1. Preserve the computational efficiency; 

2. Reduce the number of chips; 

3. Reduce the complexity of additional hardware. 

The invention is based on reducing the are a- time product by 35 
using hybrid algorithms based on a combination of Fast 
Hartley Transform (FHT) and DHT, i.e., by using higher 
radix FHT. To see this, consider again the DHT as 


where 


h=[h( 0), h( 1), h{2), . . . , h(N-l)J 

g=[g( 0), S(l), g(2), ■ ■■ , g(N-l)] 1 (26) 


Also consider the case where M=N/2. The Decimation-in- 
Frequency (DIF) of a FHT of size N in terms of DHT of size 
N/2 is given as 


fh e \(H{Nl 2) H(JV/2)Vg p ] 

UJ“U(Af/2) -S(N /2) A g r J 

(27) 

where 

55 

g P = [«(0), *(1), 


+ -.sw] 

60 


and 


h=[h{Q)Ji{2),h{4), . . . , h(N-2)Y 

65 

h 0 =[h(l),h(3)M5), ■■■, K^-i)Y 



Also, 


S(N/2)=H(N f2)K(N/2) (28) 

where K(N/2) is a matrix defined as 


^(JV/2)=Diag{cos(2jtA/JV)}+Diag{sin(23tA/AT)}P (29) 

in which k is an index that assumes an integer value from 1 
to N-l beginning at the lowest order matrix element to the 
highest order matrix element and the symbol Diag connotes 
a diagonal matrix. In the above expressions H(N/2) repre- 
sents a DHT of size N/2, and P is the permutation matrix as 
defined before. Then, Eq. (27) can be written as 

( h ‘)( fW 2 ) H WI 2 ) YM (3°) 

UJ “ { H(N 1 2)K(N 1 2) -H(N/2)K(N/2)){ gr ) 

or, equivalently 

(K\ { H(N/2)(g p + g r ) 'l < 31 ) 

UJ {H(N/2)K(N/2)(g p -g r ) i 


The process of Equation 31 is illustrated in the block flow 
diagram of FIG. 12. In FIG. 12, an adder 610 and subtractor 
620 combine the lower and higher order portions of the 
in-coming signal, g and g r , respectively, to provide a sum 
and a difference. The sum from the adder 610 is transformed 
by a processor 640 in accordance with the transform H(N/2) 
(corresponding to the top vector element of Equation 31) 
while the difference from the sub tractor 620 is transformed 
by processors 630 and 650 in accordance with the trans- 
forms K(N/2) and H(N/2), respectively (corresponding to 
the bottom vector element of Equation 31). 

Now, substituting Equation 31 into Equation 12 yields a 
DFT process which is a time-efficient and area-efficient 
realization of a DFT of size N. This process is illustrated in 
FIG. 13 as follows and may be expressed as a pair of 
equations (Equations 31 and 33 below) defining, 
respectively, the real and imaginary parts (Re and Im) of the 
even and odd components ( f e and t Q ) of the desired Fourier 
transform: 


Re 


and 


Im 


{ f ;} 

P+I 

H(N/2)(g p +g r ) 

\fo 

2 

H(N / 2)K(N / 2)(g p - g r ) y 

[ f ;] 

| P- 1 

r H(N/2)(g p +g r ) 

\fo J 

1 ~ 1 2 

< H{NI2)K{NI2){g p -g r ) j 


(32) 


(33) 


FIG. 13 illustrates a CCD/CID architecture for carrying 
out the processes of Equations 32 and 33. In FIG. 13, the 
incoming signal vector g of size N is bifurcated into two 
half-sized (N/2) portions, namely g containing the lower 
order terms of g and a portion g r containing the higher order 
terms of g. A bit-serial adder 710 computes the sum of the 
two portions of g while a bit serial adder 720 computes the 
difference between the two portions of g. The sum computed 
by the first bit -serial adder 710 is applied to the inputs of two 
N/2-size CCD/CID arrays 730, 740 of the type described 
above with reference to FIG. 4. The difference computed by 
the second bit-serial adder 720 is applied to the inputs of two 
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N/2-size CCD/CID arrays 750, 760 of the type described 
above with reference to FIG. 4. 

The CCD/CID array 730 stores charge packets corre- 
sponding to the binary values of the matrix 

5 

P+I 

~Y~ 

[H(N/2)] in respective CCD/CID cells 235 as illustrated in 
FIG. 4. The CCD/CID array 740 stores charge packets 10 
corresponding to the binary values of the matrix 

p-i 

~Y~ 

15 

[H(N/2)] in respective CCD/CID cells 235 as illustrated in 
FIG. 4. The CCD/CID array 750 stores charge packets 
corresponding to the binary values of the matrix 

20 

P+I 

~Y~ 

[ H(N/2)K(N/2) ] in respective CCD/CID cells 235 as illus- 
trated in FIG. 4. The CCD/CID array 760 stores charge 25 
packets corresponding to the binary values of the matrix 

p-i 


[ H(N/2)K(N/2) ] in respective CCD/CID cells 235 as illus- 
trated in FIG. 4. 

The CCD/CID array 730 produces the real part of the even 
terms of the Fourier transform of g. The CCD/CID array 740 35 
produces the imaginary part of the even terms of the Fourier 
transform of g. The CCD/CID array 750 produces the real 
part of the odd terms of the Fourier transform of g. The 
CCD/CID array 760 produces the Fourier transform of the 
odd terms of the Fourier transform of g. These four products 40 
together constitute the complete Fourier transform of g. 

In summary, note that, compared to the direct approach, 
the number of required CCD/CID chips has been reduced 
from 8 to 4 at the cost of only two additional simple bit-serial 
adders. Also, since the CCD/CID chip has a bit-serial data 45 
input format, performing bit-serial addition on the data will 
increase the computation time by only a cycle. Generaliza- 
tion to different M-to-N ratios is straightforward. 

While the massively parallel processes of the invention 
have been described with reference to a preferred imple- 50 
mentation employing the high-precision CCD/CID arrays of 
FIGS. 4 and 5, these processes may also be implemented 
using the prior types of CCD/CID arrays of the type illus- 
trated in FIGS. 1-3. 

While the invention has been described in detail by 55 
specific reference to preferred embodiments, it is understood 
that variations and modifications thereof may be made 
without departing from the true spirit and scope of the 
invention. 

What is claimed is: 60 

1. A charge domain computing device for performing a 
Fourier transform by simultaneous matrix-vector multipli- 
cation operations in respective plural charge coupled device/ 
charge injection device arrays comprising: 

a plurality of charge coupled device/charge injection 65 
device arrays storing charge coupled device charge 
packets in respective charge coupled device cells 


arrayed in rows and column, said charge packets having 
amounts of charge corresponding to the values of 
corresponding elements of respective matrix operators; 

a Hartley transform processor for each of said arrays 
having a signal processing operation in a matrix vector 
multiplication operation for producing real and imagi- 
nary parts of said Fourier transform using a Hartley 
transform; and 

a device for simultaneously applying an incoming vector 
to said plurality of charge coupled device/charge injec- 
tion device arrays; 

wherein each matrix-vector multiplication operation in 
respective plural charge coupled device/charge injec- 
tion device arrays simultaneously produces said real 
and imaginary parts of said Fourier transform. 

2. The device of claim 1 wherein each of said charge 
coupled device/charge injection device arrays comprises: 

an array of n rows and M columns of charge coupled 
device matrix cell groups corresponding to a matrix of 
N rows and M columns of matrix elements, each of said 
matrix elements representable with b binary bits of 
precision, each of said matrix cell groups comprising a 
column of b charge coupled device cells storing b 
charge coupled device charge packets representing the 
b binary bits of the corresponding matrix element, the 
amount of charge in each packet corresponding to one 
of two predetermined amounts of charge; 

each of said charge coupled device cells comprising a 
holding site and a charge sensing site, each charge 
packet initially residing at the respective holding site; 

a device for sensing, for each row, an analog signal 
corresponding to a total amount of charge residing 
under all charge sensing sites of the charge coupled 
device cells in the row; 

an array of c rows and M columns of charge coupled 
device vector cells corresponding to a vector of M 
elements representable with c binary bits of precision, 
each one of said M columns of charge coupled device 
vector cells storing a plurality of c charge packets 
representing the c binary bits of the corresponding 
vector element, the amount of charge in each packet 
corresponding to one of two predetermined amounts of 
charge; and 

a multiplying device operative for each one of said c rows 
of said charge coupled device vector cells for tempo- 
rarily transferring to said charge sensing site the charge 
packet in each one of said m columns of matrix cells for 
which the charge packet in the corresponding one of 
said m columns and said one row of said charge 
coupled device vector cells has an amount of charge 
corresponding to a predetermined binary value. 

3. The device of claim 2 further comprising arithmetic 
means operative in synchronism with said multiplying 
device, comprising: 

means for receiving, for each row, the signal sensed by 
said sensing device, whereby to receive Nxb signals in 
each one of c operations of said multiplying means; 

means for converting each of said signals to a correspond- 
ing byte of output binary bits; and 

means for combining the output binary bits of all of said 
signals in accordance with appropriate powers of two to 
generate bits representing an N-element vector corre- 
sponding to the product of said vector and said matrix. 

4. The device of claim 1 wherein said array of matrix 
charge coupled device cells is distributed among a plurality 
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of b integrated circuits containing sub-arrays of said M 
columns and N rows of said matrix charge coupled device 
cells, each of said sub-arrays corresponding to a bit -plane of 
matrix cells representing bits of the same power of two for 
all of said matrix elements. 5 

5. The device as set forth in claim 1, wherein the Hartley 
transform is a combination of fast Hartley transform and a 
discrete Hartley transform. 

6. A charge domain computing device for performing a 
Fourier transform by simultaneous matrix vector multipli- 10 
cation operations in respective plural charge coupled device/ 
charge injection device arrays comprising: 

a first charge coupled device/charge injection device array 
storing charge packets; 

a first Hartley transform processor for computing a first 15 
matrix operator represented by said charge packets 
derived from a first permutation of a Hartley transform 
and wherein said first array having a first matrix vector 
multiplication operation which produces the real part of 
the Fourier transform of an incoming vector; 

a second charge coupled device/charge injection device 
array storing charge packets; 

a second Hartley transform processor for computing a 
second matrix operator represented by charge packets 2 s 
derived from a second permutation of said Hartley 
transform and having a second matrix vector multipli- 
cation operation which produces the imaginary part of 
the Fourier transform of said incoming vector; and 

a device for simultaneously applying said incoming vec- 30 
tor to said first and second charge coupled device/ 
charge injection device arrays; 

wherein said first and second matrix-vector multiplication 
operations in respective plural charge coupled device/ 
charge injection device arrays simultaneously produce 35 
said respective real and said imaginary parts of said 
Fourier transform. 

7. The device of claim 6 wherein said first permutation 
corresponds to a sum of permutated and unpermutated 
version of said Hartley transform and said second permu- 40 
tation corresponds to a difference between said permutated 
and unpermutated versions of said Hartley transform. 

8. The device of claim 6 wherein said Hartley transforms 
and said charge coupled device/charge injection device 
arrays are each of a size m which is a fraction of the size N 45 
of the Fourier transform and wherein said incoming vector 

is divisible into smaller vectors combinable into respective 
sums and differences and wherein said device for applying 
applies respective ones of said sums and differences to 
respective ones of said charge coupled device/charge injec- 50 
tion device arrays. 

9. The device of claim 8 wherein said incoming vector is 
divisible into two equal portions, each one of said first and 
second charge coupled device/charge injection device arrays 
comprising a pair of individual charge coupled device/ 55 
charge injection device arrays of size M, four different 
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permutations of said Hartley transform being embedded in 
the respective M-size charge coupled device/charge injec- 
tion device arrays. 

10. The device of claim 6 wherein each of said charge 
coupled device/charge injection device arrays comprises: 

an array of N rows and M columns of charge coupled 
device matrix cell groups corresponding to a matrix of 
N rows and M columns of matrix elements, each of said 
matrix elements representable with b binary bits of 
precision, each of said matrix cell groups comprising a 
column of b charge coupled device cells storing b 
charge coupled device charge packets representing the 
b binary bits of the corresponding matrix element, the 
amount of charge in each packet corresponding to one 
of two predetermined amounts of charge; 
each of said charge coupled device cells comprising a 
holding site and a charge sensing site, each charge 
packet initially residing at the respective holding site; 
a device for sensing, for each row, an analog signal 
corresponding to a total amount of charge residing 
under all charge sensing sites of the charge coupled 
device cells in the row; 

an array of c rows and M columns charge coupled device 
vector cells corresponding to a vector of M elements 
representable with c binary bits of precision, each one 
of M columns of charge coupled device vector cells 
storing a plurality of c charge packets representing c 
binary bits of the corresponding vector element, the 
amount of charge in each packet corresponding to one 
of two predetermined amounts of charge; and 
a multiplying device operative on each one of said c rows 
of said charge coupled device vector cells for tempo- 
rarily transferring to said charge sensing site the charge 
packet in each one of said M columns of matrix cells 
for which the charge packet in the corresponding one of 
said M columns and said one row of said charge 
coupled device vector cells has an amount of charge 
corresponding to a predetermined binary value. 

11. The device of claim 10 further comprising arithmetic 
means operative in synchronism with said multiplying 
device, comprising: 

means for receiving, for each row, the signal based by said 
sensing device, whereby to receive Nxb signals in each 
one of c operations of said multiplying means; 
means for converting each of said signals to a correspond- 
ing byte of output binary bits; and 
means for combining the output binary bits of all of said 
signals in accordance with appropriate powers of two to 
generate bits representing an N-element vector corre- 
sponding to the product of said vector and said matrix. 

12. The device as set forth in claim 6, wherein the Hartley 
transform is a combination of a fast Hartley transform and 
a discrete Hartley transform. 



